Backpropagation for long sequences: beyond memory constraints with constant overheads

Naive backpropagation through time has a memory footprint that grows linearly in the sequence length, due to the need to store each state of the forward propagation. This is a problem for large networks. Strategies have been developed to trade memory for added computations, which results in a sublinear growth of memory footprint or computation overhead. In this work, we present a library that uses asynchronous storing and prefetching to move data to and from slow and cheap stor- age. The library only stores and prefetches states as frequently as possible without delaying the computation, and uses the optimal Revolve backpropagation strategy for the computations in between. The memory footprint of the backpropagation can thus be reduced to any size (e.g. to fit into DRAM), while the computational overhead is constant in the sequence length, and only depends on the ratio between compute and transfer times on a given hardware. We show in experiments that by exploiting asyncronous data transfer, our strategy is always at least as fast, and usually faster than the previously studied "optimal" strategies

Spud 1.0: generalising and automating the user interfaces of scientific computer models

The interfaces by which users specify the scenarios to be simulated by scientific computer models are frequently primitive, under-documented and ad-hoc text files which make using the model in question difficult and error-prone and significantly increase the development cost of the model. In this paper, we present a model-independent system, Spud, which formalises the specification of model input formats in terms of formal grammars. This is combined with an automated graphical user interface which guides users to create valid model inputs based on the grammar provided, and a generic options reading module, libspud, which minimises the development cost of adding model options. <br><br> Together, this provides a user friendly, well documented, self validating user interface which is applicable to a wide range of scientific models and which minimises the developer input required to maintain and extend the model interface

Shoreline and Bathymetry Approximation in Mesh Generation for Tidal Renewable Simulations

Due to the fractal nature of the domain geometry in geophysical flow simulations, a completely accurate description of the domain in terms of a computational mesh is frequently deemed infeasible. Shoreline and bathymetry simplification methods are used to remove small scale details in the geometry, particularly in areas away from the region of interest. To that end, a novel method for shoreline and bathymetry simplification is presented. Existing shoreline simplification methods typically remove points if the resultant geometry satisfies particular geometric criteria. Bathymetry is usually simplified using traditional filtering techniques, that remove unwanted Fourier modes. Principal Component Analysis (PCA) has been used in other fields to isolate small-scale structures from larger scale coherent features in a robust way, underpinned by a rigorous but simple mathematical framework. Here we present a method based on principal component analysis aimed towards simplification of shorelines and bathymetry. We present the algorithm in detail and show simplified shorelines and bathymetry in the wider region around the North Sea. Finally, the methods are used in the context of unstructured mesh generation aimed at tidal resource assessment simulations in the coastal regions around the UK

Temporal blocking of finite-difference stencil operators with sparse "off-the-grid" sources

Stencil kernels dominate a range of scientific applications, including seismic and medical imaging, image processing, and neural networks. Temporal blocking is a performance optimization that aims to reduce the required memory bandwidth of stencil computations by re-using data from the cache for multiple time steps. It has already been shown to be beneficial for this class of algorithms. However, applying temporal blocking to practical applications' stencils remains challenging. These computations often consist of sparsely located operators not aligned with the computational grid (“off-the-grid”). Our work is motivated by modelling problems in which source injections result in wavefields that must then be measured at receivers by interpolation from the grided wavefield. The resulting data dependencies make the adoption of temporal blocking much more challenging. We propose a methodology to inspect these data dependencies and reorder the computation, leading to performance gains in stencil codes where temporal blocking has not been applicable. We implement this novel scheme in the Devito domain-specific compiler toolchain. Devito implements a domain-specific language embedded in Python to generate optimized partial differential equation solvers using the finite-difference method from high-level symbolic problem definitions. We evaluate our scheme using isotropic acoustic, anisotropic acoustic, and isotropic elastic wave propagators of industrial significance. After auto-tuning, performance evaluation shows that this enables substantial performance improvement through temporal blocking over highly-optimized vectorized spatially-blocked code of up to 1.6x

Intra-arterial nitroglycerin as directed acute treatment in experimental ischemic stroke

BACKGROUND: Nitroglycerin (also known as glyceryl trinitrate (GTN)), a vasodilator best known for treatment of ischemic heart disease, has also been investigated for its potential therapeutic benefit in ischemic stroke. The completed Efficacy of Nitric Oxide in Stroke trial suggested that GTN has therapeutic benefit with acute (within 6 hours) transdermal systemic sustained release therapy. OBJECTIVE: To examine an alternative use of GTN as an acute therapy for ischemic stroke following successful recanalization. METHODS: We administered GTN IA following transient middle cerebral artery occlusion in mice. Because no standard dose of GTN is available following emergent large vessel occlusion, we performed a dose-response (3.12, 6.25, 12.5, and 25 µg/µL) analysis. Next, we looked at blood perfusion (flow) through the middle cerebral artery using laser Doppler flowmetry. Functional outcomes, including forced motor movement rotor rod, were assessed in the 3.12, 6.25, and 12.5 µg/µL groups. Histological analysis was performed using cresyl violet for infarct volume, and glial fibrillary activating protein (GFAP) and NeuN immunohistochemistry for astrocyte activation and mature neuron survival, respectively. RESULTS: Overall, we found that acute post-stroke IA GTN had little effect on vessel dilatation after 15 min. Functional analysis showed a significant difference between GTN (3.12 and 6.25 µg/µL) and control at post-stroke day 1. Histological measures showed a significant reduction in infarct volume and GFAP immunoreactivity and a significant increase in NeuN. CONCLUSIONS: These results demonstrate that acute IA GTN is neuroprotective in experimental ischemic stroke and warrants further study as a potentially new stroke therapy

Architecture and performance of Devito, a system for automated stencil computation

Stencil computations are a key part of many high-performance computing applications, such as image processing, convolutional neural networks, and finite-difference solvers for partial differential equations. Devito is a framework capable of generating highly-optimized code given symbolic equations expressed in Python, specialized in, but not limited to, affine (stencil) codes. The lowering process -- from mathematical equations down to C++ code -- is performed by the Devito compiler through a series of intermediate representations. Several performance optimizations are introduced, including advanced common sub-expressions elimination, tiling and parallelization. Some of these are obtained through well-established stencil optimizers, integrated in the back-end of the Devito compiler. The architecture of the Devito compiler, as well as the performance optimizations that are applied when generating code, are presented. The effectiveness of such performance optimizations is demonstrated using operators drawn from seismic imaging applications

Survey of the needs of patients with spinal cord injury: impact and priority for improvement in hand function in tetraplegics\ud

Objective: To investigate the impact of upper extremity deficit in subjects with tetraplegia.\ud \ud Setting: The United Kingdom and The Netherlands.\ud \ud Study design: Survey among the members of the Dutch and UK Spinal Cord Injury (SCI) Associations.\ud \ud Main outcome parameter: Indication of expected improvement in quality of life (QOL) on a 5-point scale in relation to improvement in hand function and seven other SCI-related impairments.\ud \ud Results: In all, 565 subjects with tetraplegia returned the questionnaire (overall response of 42%). Results in the Dutch and the UK group were comparable. A total of 77% of the tetraplegics expected an important or very important improvement in QOL if their hand function improved. This is comparable to their expectations with regard to improvement in bladder and bowel function. All other items were scored lower.\ud \ud Conclusion: This is the first study in which the impact of upper extremity impairment has been assessed in a large sample of tetraplegic subjects and compared to other SCI-related impairments that have a major impact on the life of subjects with SCI. The present study indicates a high impact as well as a high priority for improvement in hand function in tetraplegics.\ud \u

Detection of myxoma viruses encoding a defective M135R gene from clinical cases of myxomatosis; possible implications for the role of the M135R protein as a virulence factor

<p>Abstract</p> <p>Background</p> <p>Myxoma virus is a member of the <it>Poxviridae </it>and causes disease in European rabbits. Laboratory confirmation of the clinical disease, which occurs in the autumn of most years in Denmark, has been achieved previously using antigen ELISA and electron microscopy.</p> <p>Results</p> <p>An unusually large number of clinically suspected cases of myxomatosis were observed in Denmark during 2007. Myxoma virus DNA was detected, using a new real time PCR assay which targets the M029L gene, in over 70% of the clinical samples submitted for laboratory confirmation. Unexpectedly, further analysis revealed that a high proportion of these viral DNA preparations contained a frame-shift mutation within the M135R gene that has previously been identified as a virulence factor. This frame-shift mutation results in expression of a greatly truncated product. The same frame-shift mutation has also been found recently within an avirulent strain of myxoma virus (6918). However, three other frame-shift mutations found in this strain (in the genes M009L, M036L and M148R) were not shared with the Danish viruses but a single nucleotide deletion in the M138R/M139R intergenic region was a common feature.</p> <p>Conclusions</p> <p>It appears that expression of the full-length myxoma virus M135R protein is not required for virulence in rabbits. Hence, the frame-shift mutation in the M135R gene in the nonpathogenic 6918 virus strain is not sufficient to explain the attenuation of this myxoma virus but one/some of the other frame-shift mutations alone or in conjunction with one/some of the thirty two amino acid substitutions must also contribute. The real time PCR assay for myxoma virus is a useful diagnostic tool for laboratory confirmation of suspected cases of myxomatosis.</p

Over-Expression of a Cytochrome P450 Is Associated with Resistance to Pyriproxyfen in the Greenhouse Whitefly Trialeurodes vaporariorum

Copyright: 2012 Karatolos et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Background: The juvenile hormone mimic, pyriproxyfen is a suppressor of insect embryogenesis and development, and is effective at controlling pests such as the greenhouse whitefly Trialeurodes vaporariorum (Westwood) which are resistant to other chemical classes of insecticides. Although there are reports of insects evolving resistance to pyriproxyfen, the underlying resistance mechanism(s) are poorly understood. Results: Bioassays against eggs of a German (TV8) population of T. vaporariorum revealed a moderate level (21-fold) of resistance to pyriproxyfen. This is the first time that pyriproxyfen resistance has been confirmed in this species. Sequential selection of TV8 rapidly generated a strain (TV8pyrsel) displaying a much higher resistance ratio (>4000-fold). The enzyme inhibitor piperonyl butoxide (PBO) suppressed this increased resistance, indicating that it was primarily mediated via metabolic detoxification. Microarray analysis identified a number of significantly over-expressed genes in TV8pyrsel as candidates for a role in resistance including cytochrome-P450 dependent monooxygenases (P450s). Quantitative PCR highlighted a single P450 gene (CYP4G61) that was highly over-expressed (81.7-fold) in TV8pyrsel. Conclusion: Over-expression of a single cytochrome P450 gene (CYP4G61) has emerged as a strong candidate for causing the enhanced resistance phenotype. Further work is needed to confirm the role of the encoded P450 enzyme CYP4G61 in detoxifying pyriproxyfen.Peer reviewedFinal Published versio